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Abstract 

Many virtual worlds have been adopted for implementation within educational settings 
because they are potentially useful for building effective learning environments. Since the 
flexibility of virtual worlds challenges to obtain effective and efficient educational 
outcomes, the design of such platforms need more attention. In the present study, the aim 
was to investigate design issues and the concerns of novice instructional designers who 
were developing a virtual learning environment with a gaming approach. Active Worlds 
(AW), an immersive virtual environment, was selected as a platform to develop virtual world 
projects that presented instructional materials. Twenty-three novice instructional designers 
participated in this study. Data were collected through observations, interviews, document 
analysis, and a questionnaire. The results indicate that the novice instructional designers 
were most challenged when attempting to incorporate motivational and assessment 
features in their project designs. They ignored collaboration opportunities of the design 
platform. They also accepted that they could not achieve to provide a feel of real game. The 
results suggested that design and development in virtual worlds has many challenges but a 
broad experience for novice instructional designers. 

Keywords: Instructional design; Novice instructional designers; Instructional design 
education; Virtual worlds; Active Worlds 


Introduction 

In recent years, the use of virtual worlds in education has increased dramatically, as they have 
been claimed to be effective settings for distance education and collaborative learning. These 
environments engage users by allowing them to partly create their own learning settings and 
then to play a role within those settings. Liu and Rutledge (1996) emphasized that designing an 
interactive environment is a very arduous and complex process. Because the aim of interactive 
media is to offer more than static information, designers must consider interaction features, help 
and feedback components, and pedagogical and graphics aspects of the software (Papert & Harel 
1991). With a number of affordances, virtual worlds allow many instructional methods to be 
applied (Amichai-Hamburger & McKenna, 2006; Barab, Thomas, Dodge, Carteaux & Tuzun, 
2005; Berns, Gonzalez-Pardo & Camacho, 2013; Duncan, Miller & Jiang, 2012; Kim, Lee & 
Thomas, 2012; Okutsu, DeLaurentis, Brophy & Lambert, 2013; Soukap, 2004). On the other 
hand, virtual worlds have a flexible structure that might cause learners lost their attention or 
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confuse on which piece of knowledge that they need to focus (Dickey, 2005a; Ho, Rappa & Chee, 
2009). Therefore, more structured and goal-based designs might be helpful to engage students 
learning. 

In this respect, game based approach might fit well in virtual worlds with its fantastic graphics, 
interactive objects and communication tools. However, it must be investigated that what turns 
virtual worlds to a game and whether there is any challenge while creating a game-like learning 
environment in virtual world. This study was designed to reveal what sorts of challenges arose 
while instructional designers were attempting to design a virtual world educational setting in a 
gamified way and to examine their design decisions. The results of the study will contribute to 
our understanding of instructional designers' challenges while approaching a new case of 
instructional design and how a virtual world might become a game based environment via 
instructional designers' decisions. 


Background of the Study 

Virtual worlds have become one of the most important platform types among computer-based 
learning environments. In the last decade, virtual worlds proliferated because they were used 
for distance education as well as socialization. Many virtual world platforms allow users to 
create learning environments. For example, in the Active Worlds (AW) platform there are more 
than 600 large-scale worlds, each with its own server (Active Worlds, 2016). Thousands of other 
types of learning platforms also exist on the main server of AW (Active Worlds 2016). Such virtual 
worlds accommodating various type of knowledge representation have changed our perception 
of learning environments (Dickey 2005a). Those environments changed the profile of the 
younger generation of students, who now have different demands than older learners (Tuzun, 
2007). Because younger learners demand game-like environments, educators have begun to see 
virtual environments as motivational learning tools. 

Although virtual worlds are not real games, they allow users to design game like environments. 
Virtual worlds also allow learners from different cultures or countries to collaborate and 
communicate. In addition, this type of environment utilizes learner-centered strategies 
(Kebritchi & Hirumi, 2008), as well as independent (Dickey, 2005a) and self-directed learning 
(Heid & Kretschmer, 2009). Yet another benefit is that virtual worlds offer learners real-world 
experiences within virtual communities (Nelson & Erlandson, 2008). Virtual worlds can be 
developed in various approaches and for different purposes including exhibition areas (like 
museum), real classrooms, immersive demonstrations, role playing environment, simulation of 
events or experiments, animation of historical events and places, task based activities (Barab, 
Dodge, Tuzun, Job-Sluder, Jackson & A. Arici et al., 2008; Brom, Preuss & Klement, 2011; Kay & 
Fitzgerald, 2008). Therefore, they might be considered as one of the option for instructional 
designers while designing educational platforms. Thus, it is worth to investigate their 
shortcomings as an educational platform. 

Each of those game-like and simulated environments developed in virtual worlds has a different 
strategy and design (Hirumi et al., 2010b). However, though the number of available applications 
has increased, their design involves several technical and pedagogical challenges that may limit 
what instructional designers can achieve with these systems. For example, Kemp and 
Livingstone (2006) observed that the lack of interactivity options in Second Life negatively 
affected the integration of its learning management systems with its immersive environment. 
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After experiencing a virtual zoo project, Wheeler (2009) observed some technical problems 
regarding its communication methods, development time, and activities. In their study, Coban, 
Karakus, Karaman, Gunay and Goktas (2015) showed that an effective interaction is very 
challenging the programmers when they want to trigger any object by the movement or hit of 
another object (like moving a ball after hitting it with a stick). Revealing those kind of issues 
might be helpful for educators and designers who need to design game-like environments. 

According to Ertmer et al. (2009), novice instructional designers tend to take information at face 
value, raise issues in order to blame others, focus on things they do not understand, and search 
for information mechanically. Despite these unfortunate tendencies, Horn and Masunaga (2006) 
argued that abilities can be enhanced with deliberate practice. While making judgments about 
the context, scientific guidance is not adequate for novice instructional designers since each 
educational context has many different dynamics (Gray et al., 2015). Therefore, in Instructional 
Design education, novices should be exposed to a wide variety of design context, in order to gain 
experience with multiple platform types and to find innovative ways to design them for learning 
purposes (Hirumi et al., 2010b). 

Interactive media design training employs an immersion strategy and offers novice instructional 
designers a structure problem to solve collaboratively (Dabbagh & Blijd, 2010). Cheney et al. 
(2009) stated that whenever a new productive, social, and collaborative technology is 
incorporated into a curriculum, this always generates challenges to find the best way to use 
these for instruction. The way by which instructional designers solve design problems and apply 
knowledge, the processes they employ, the goals they establish, and their management, 
monitoring, and evaluation processes are reflection of their instructional design skills 
(Gustafson & Branch 2007). To understand how novice instructional designers gain expertise in 
Instructional Design (ID), their decision-making processes, methods, and products must be 
examined (Hardre, Ge & Thomas 2006). Novice instructional designers who are faced with 
design problems need support (Hardre et al., 2006) and explicit guidelines that are based on 
expert thinking (Ertmer et al., 2009). In order to best provide that support, ID educators must 
understand their students' approaches to problems. 

In this regard, virtual worlds are an ideal testing ground for studying the decisions of novice 
designers. Both technical and pedagogical issues arise when designing virtual worlds to be used 
as learning environments.Virtual worlds are not technically difficult to design, but some of their 
features or limitations may not be ideally suited to implanting pedagogical components (Hirumi 
2010a). This study was designed to reveal critical issues that influence novice instructional 
designers' design processes regarding the creation of game-like educational environments in 
virtual worlds. Specifically, this study investigated the following research questions: 

1. What are the challenges of novice instructional designers who design and develop 
virtual worlds for educational purposes? 

2. What are the design decisions of novice instructional designers to integrate game-like 
learning experience into virtual worlds? 

Active Worlds (AW), an immersive virtual environment, was selected for use in this study. The 
participants were novice instructional designers, who were given the task of creating learning 
environments for different target audiences. The challenges that they faced regarding both 
instructional and technological features and their solutions were examined. 
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Significance of the Study 

This study is important in several aspects. First, as stated previously, virtual worlds are popular 
platforms in educational settings because of their utility for creating effective learning 
environments. Transforming these environments into game-like platforms might be challenging. 
However, virtual environment must provide easy way to create better learning conditions 
without advance technical knowledge (Munoz-Cristobal, Prieto, Asensio-Perez, Martinez- 
Mones, Jorrin-Abellan, & Dimitriadis, 2015). This study exemplifies how game elements might 
be integrated into the design to create game like environment. Thus, educators might facilitate 
in providing a game experience with virtual worlds without advance technical background. 

Thinking that any maestro needs to know to play some instruments to make a perfect 
orchestration, novice instructional designers need to gain both technical and pedagogical 
experience to be able to choose the optimum tools for different kind of educational purposes. 
Instructional designers must be familiar with the design and development process for such kind 
of environments. Studying the perspectives of novice instructional designers, and the challenges 
which they face is important, so that ID instruction can be better adapted to help novices to 
perform more efficiently in the future. Working with virtual worlds also offers the opportunity 
to test novices' competencies over a wide range of areas. Lastly, although virtual worlds have 
many advantages they still have some technical issues that might lead pedagogical inadequacies. 
Results of the study will help to guide virtual world developers to improve the pedagogical and 
technical features of these platforms. 

As a last issue, this study exemplifies in which part of instruction, instructional designers 
experience challenges. The challenges that novices faced might be considered as the general 
issues in similar design platforms. Therefore, the study might give some insights about where 
the instructors of instructional design courses provide scaffolding. 


Methodology 

In this qualitative case study, researchers aimed to understand the challenges and design 
decisions of novice instructional designers while designing game like learning environment. 
Therefore, the context was an undergraduate instructional design course, which senior students 
enrolled. 


Participants 

The participants in the study were 23 undergraduate Instructional Technology students studying 
at a high ranked university in the capital city of Turkey, 8 females and 15 males, who were in 
their senior year and enrolled in an educational software development course. New technology 
has been integrated into the curriculum, as well as programming, instructional design, and 
pedagogical courses. Graduates of the department typically find subsequent work as 
instructional designers, computer teachers, programmers, or system analysts in public or private 
institutions. The participating students formed seven different project groups. Each researcher 
selected two or three groups randomly to conduct deep observations and interviews during 
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regular meetings. The students were free to form their groups, and the groups were not 
homogenous. Table 1 shows the group details and their respective project topics. 

Table 1. Facilitators and Project Group Gender Distributions 


Facilitator 1 

Group 1 - Installing Computer Networks 
Group 2 - Exploring the Library 

Group 3 - Exploring the Health Center 

Two females, two males 

Three females, one male 

Three males 

Facilitator 2 

Group 4 - Exploring the Library 

Three males 


Group 5 - Biology (Cell) 

Three males 

Facilitator 3 

Group 6 - Exploring the Museum 

Three females 


Group 7 -Exploring the Basic English Dept. 

Three males 


The participants are called as novice instructional designers because starting the freshmen class, 
they have taken instructional design courses and in most of them they worked with a real target 
group. For example, in "Instructional Design" and "Multimedia Design and Development" 
courses, they worked with elementary students to develop materials for Science Education. 
They also experienced in web design, graphics and animation design, distance education, project 
development and educational sciences. 


The Setting 

The study was conducted in the context of an educational software design course. The study 
lasted for 14 weeks. Each week, the students attended a traditional class (2 hours) and met with 
their facilitators to discuss their projects. Besides, for the first four weeks, they attended lab 
sessions (2 hours per week). In the first two lab sessions, they played several computer games 
and wrote reflections about the educational aspects of those games. Those games were 
Dimenxian (a 3D game for learning coordinate system), Contraptions (A 2D puzzle game based 
on physical rules), a modified Tomb Raider game for learning biology. Those games were 
selected by instructor and course teaching assistants because of their educational aspects and 
variety of game genres. For the latter two weeks of their lab work, teaching assistants introduced 
Active Worlds (AW), which is the 3D virtual world development platform. They explained 
technical features, how the student should access it, how to design within it, and how to make 
connections between objects and Web components. Each group obtained a basic account from 
Active Worlds. With this account, they could use available objects and build their learning 
environments. 

In order to design and develop their game-like virtual worlds, a particular instructional design 
model was followed by the students. This was the FID 2 GE, which stands for Fuzzified 
Instructional Design Development of Game-like Environments (Kaplan-Akilli & Cagiltay 2007). 
The model includes intertwined analysis, design and development, implementation, and 
evaluation phases. In the analysis stage, the participants defined their goals; reported the results 
of needs, content, context, learner, tool (AW) and game analysis; defined their instructional 
approach; and adjusted it to their design. In the design and development report, they reported 
all of the details of their design, along with their motivational, assessment, and feedback 
decisions; defined their clear-cut scenarios; adjusted their scenarios with their instructional 
approaches; reported maintenance and distribution issues; and presented an introduction 
about implementation and evaluation issues. The students also prepared short reports on their 
pilot studies, for which they used both paper-based and computer-based prototypes. In the 
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implementation phase, they evaluated their products regarding whether they had met their 
project goals, and they eliminated usability problems. They reported on this phase in a final 
report. In that final report, they discussed changes made in their scenarios and the plans they 
formulated during the development process. 

All of the groups were encouraged to develop a game-like environment to introduce one part of 
the university campus (the library, medical center, museum etc.) to people who are new to the 
university. So in evaluation of each phase they need to work with freshmen. Two of the groups 
preferred to develop different environments, which focused on biology and computer networks. 

In the AW setting, each group had a particular virtual place to create their own world. The AW 
user interface includes three parts. In the interactive 3D interface part, the students created 3D 
objects and wrote simple scripts to make them perform a task. Users can walk and move in the 
3D environment, and see those objects. They can also interact with them by clicking on the 
objects. The second part of the interface is a Web browser with which students may upload or 
download a variety of resources, or interact with databases. When users click on an object, the 
content of the browser might change, if it was coded. Most of time, this feature provides 
information about the objects or events that occur in that particular area of the virtual world. 
Lastly, the AW includes a chat tool that allows real-time communication with the other AW 
users. Users can chat with other players or with any support person. Users must be in the same 
virtual world to see each other and their messages. A snapshot of the AW entrance is shown in 
Figure 1. 
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Figure 1. A Snapshot of an AW Window 


Instruments and Data Gathering Procedures 

The data collection process took three months. This three-month period was divided into the 
analysis, design and development, and final stages in accordance with the structure of the 
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course. The researchers collected a total of 24 group-meeting observation notes during the 
semester. Each meeting was lasted about half an hour. The researchers participated in all of the 
classes and used their observations from these sessions. An additional special session was also 
held to discuss virtual world issues. The groups wrote their first group reflections based on their 
AW experiences. Then, the class discussed the experiences. The groups' reflection papers about 
the tool were collected as an instrument, and the group discussion was recorded in that session. 

At the beginning of the semester, the groups were given an instructional design questionnaire 
related to their projects. The groups answered several questions, such as: How did they form 
their project team; how did they select a pilot study site; what kind of tools did they use; what 
kind of model did they choose; etc. The answers from the seven groups were collected and 
analyzed. Their answers about selecting an instructional environment development tool and 
about issues that they took into consideration during the planning process were also evaluated. 

Toward the end of the semester, an open-ended questionnaire that inquired about their 
instructional design decisions was distributed to the students. The questions focused on the 
materials, the tool used to develop those materials, and the strategies that the students used to 
design the instructional elements. 

At the end of the semester, voluntary students were invited to participate in individual 
interviews, and six students participated in these interviews. 

Each of the seven groups submitted three reports on their projects during the semester: the 
analysis, design, and final reports. These 21 reports were examined to analyze the participants' 
design decisions, their explanations concerning motivational issues, scenarios, and their 
interactions with the tool (AW). All of the reports were written considering a report template, 
which was prepared by instructor. Template titles were definite but what groups had written 
under the titles was flexible. Reports of project groups were also used to see what novices 
designed in their minds and how much of that designs was put into practice in the development 
platform. The researchers compared the draft of the projects on the reports and the projects 
developed on the virtual world. Table 2 displays the project development process and data 
collection times. 

Table 2. Project Development Process and Data Collection 


Term 

Data Collection Tool 

Total Data 


Group meetings 

6 group meeting records 

Analysis 

Observations 

3 class observation notes 

Analysis Report (A) 

7 group reports 


Questionnaire 

23 students 

Design and 

Group meetings 

15 group meeting records 

Development 

Observations 

5 observation notes 


Design & Development Report (D) 

7 group reports 


Group meetings 

15 group meeting records 


Observations 

4 observation notes 

Final Stage 

Individual interviews 

6 volunteers 


Final Report (F) 

7 group reports 


Developed game like learning environments 

Products of 7 groups 
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Data Analysis 

All of the interviews, observation notes, the open-ended questionnaire, and the reports were 
coded by two researchers. Both researchers were experienced with the course and qualitative 
studies. In the pilot study the inter-reliability of the data were like below (see Table 3). 

Table 3. Inter-reliability Scores of Each Instrument 


Type of the data 

Percentage of agreement 

Observation 

.80 

Questionnaire 

.82 

Reports 

.63 

Group meeting interviews 

.83 

Individual interviews 

.83 


During the observations and group meetings, the researchers realized that the students' 
concerns tended to mainly revolve around instructional, motivational, and assessment issues, 
and also the technical challenges of the AW environment. Therefore, these main titles and issues 
defined the basic codes. Each main title was evaluated in terms of the challenges posed, the 
advantages seen while using AW as a distance learning tool, and how the novices dealt with 
these issues during their project. 


Findings 

The results showed that there were three issues in game development; integration of the 
learning content with game, motivation and fun factors, feedback and rewarding. For each issue, 
both of the research questions are addressed. 


Integrating Learning Content with Game 

Most of the project groups developed their learning environment on the structural basis of drill 
and practice. In fact, most of them initially attempted to design their scenarios as a real game; 
but after a while, they realized that it was impossible to program this 3D environment as a real 
game. Then, all of the project groups used the goal-based scenario model of Schank et al. (2001) 
to create a game like experience. They created a story about the project topic and presented a 
mission to the user. While the novice instructional designers were designing missions for the 
users, they planned the instructional materials to appear in a step-by-step manner to limit the 
users' actions. This means that the users had to follow a specific route to investigate and solve 
problems in the 3D world. To guide the players, the students designed several short messages 
and questions in the 3D and the Web settings of the project. Some project groups also used 
additional paper-based materials to obtain background knowledge from the users and to control 
the users' actions. 

Table 4 shows in which novice instructional designers challenged in integrating learning content 
with the game and which strategies they used to overcome this issue. 
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Table 4. Instructional Issues that Arose During the Projects 


Challenges in creating a game with learning content 

Lack of content interactivity in the immersive part 
Split attention between different parts of the environment 
Difficulty concerning the learner-centered design 
Instructional integration strategies 
Drill and practice questions 
Goal-based scenarios 
Step-by-step routing 
Using short messages and questions 
Using additional paper-based materials 


The novices had some problems integrating the Web component with the 3D setting of the 
environment. Although the users moved around in a 3D environment (as avatars) and 
collaborated with other people, they had few opportunities to interact with 3D objects. For 
example, interacting in a virtual chemistry laboratory does not mean that the players were able 
to hold some virtual chemical substances and mix them to see the results. The novice designers 
had to use the Web feature of AW to inform the users about their progress. Thus, they used 
more than the 3D setting to provide content-learner interaction. Nowadays, immersive 
platforms have provided 3D objects (like big screens), in which a user can open Web browsers 
within the virtual platform. However, even in this case, a user would need to search Web 
browser objects by walking around the 3D platform to see the Web content. 

Generally, projects based on virtual worlds are designed with goal-based scenarios, and users 
are supposed to solve problems in the world by searching within an immersive environment, on 
Internet search engines, or using other kinds of sources. Thus, a learner-centered design could 
be achieved, because the learners have the opportunity to use a wide variety of sources to 
create their own answers. However, in the projects in this study, the participants tried to 
develop environments in which the users could find all of the resources that they needed to 
solve a problem within that learning environment. This might be an easy way for learners to 
play; but it also limited them to the content in the learning environment, and they had to follow 
a specific path to reach the learning objectives. 

By means of the chat window, the players could talk to other people, and within the immersive 
environment, they saw each other as avatars. In the group reports, although collaboration issues 
were always emphasized, the groups never incorporated collaborative learning strategies into 
the designs of their projects. In their designs, the players were conceived as individuals, and 
interaction and knowledge-sharing activities were ignored. The participating novice designers 
might have ignored this because of the AW's natural collaborative structure, which relies upon 
avatars and the chat tool. However, a collaborative learning strategy should cause the users to 
compete or interact with each other to achieve their learning objectives. In this environment, 
due to the system limitations, two avatars (representing real players) cannot interact, compete, 
or collaborate spatially. Thus, the participating designers did not attempt such collaboration, 
because it was not possible technically. Another reason might be that the designers wanted to 
control all of the actions of the users automatically. They may have believed that even if they 
could provide interaction, they could not monitor the users' interactions to provide appropriate 
feedback or to evaluate them. 
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Designing Motivation and Fun Factors 

One of the most important motivational generators that the novice designers sought to program 
into their virtual learning environments was competition. This competition could be against 
either a time limitation or the other players. Learner control was another feature that the novice 
designers emphasized. They wanted to provide several choices and challenges to the users to 
permit this learner control. 

To provide motivation in their designed environments, some project groups preferred to use 
fantastic stories, even though their target groups were mostly higher-education students. For 
example, for the university Health Center game project, the students created a story about an 
epidemic occurring on campus, and they told the users that they should suspect that they are 
ill. Their mission was to find out whether the user was sick or not while exploring the Health 
Center. The project group which developed a computer network installation guide used a real- 
life story based upon the premise that the players want to apply for an information technology 
job. To convince the boss of their qualifications, the players had to install a network. To design 
an environment which looks like a game, the groups used different strategies. For example, to 
pass from one place to another or to collect some objects, the players had to enter a number, 
which was shown in the 3D setting, into a textbox shown using the Web feature. Or, they 
provided some questions and feedback on a Web page. Thus, the novice designers tried to use 
the Web feature actively. In some projects, the learners could see their progress posted on a 
Web page, on which the designers provided some colorful objects to show which missions were 
completed. Table 5 displays a summary of the motivational and fun elements. 

Table 5. Motivation and Fun Elements Used in the Projects 


Challenges in creating motivation and fun elements 

Difficult to provide a challenge 
Difficult to give a reward in the 3D setting 
No feeling of play 

Lack of opportunities to utilize cultural differences 

Choosing a topic that could be attractive in the immersive environment 

Integration of motivation and fun elements 

Extraordinary stories 
Real-life stories 

Providing game-like activities via the Web feature 
Showing progress of students via the Web feature 


The novice designers believed that challenge was an important strategy to provide motivation 
in the 3D setting, but they neglected to use cognitive challenge. For example, in the report of 
the Library project group, they said they made the users explore the entire library to find three 
hidden books on virtual shelves which were placed very far away from each other. Thus, the 
designers could provide a challenge spatially, but not cognitively. Although the virtual worlds do 
not limit the movement and playing time of the users, one of the project groups wanted to 
incorporate a time limitation. They were able to do this with the Web feature, but it was not 
possible to program a time limitation into the 3D setting of the game. 

Almost all of the novice designers reported that the Web feature dependency was a drawback 
for motivation. In one of the observation sessions, a student reflected on this and said, the 
"immersive part is nothing without the 2D part [Web part]. Players just walk around in the virtual 
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world if they are not engaged in some operations by means of the 2D part." Another student 
supported this view and said, “...our game project could not provide real entertainment. This is 
not a game in our minds, and to make it educational, we have to use the Web part." Before 
starting the projects, the novice designers thought that they could develop a game in the design 
platform. During the individual interviews, one of the students said: "... we are going to schools 
for teaching practice. The 10 th and 11 th graders do nothing. They just play Flash-based games. 
They do not want to participate in computer lessons, and it is difficult to make them concentrate 
in this environment [AW project]... This is the biggest handicap of this environment." The novice 
designers' concerns about motivation were especially focused upon engagement. On the other 
hand, after encountering some design examples, they admitted that the AW environment can 
provide a fantastic environment. They particularly liked the quality and variety of the objects. 


Designing Feedback and Rewarding Issues 

As seen in Table 6, the novice designers had to provide a student assessment model in their 
designs. Assessment was assumed as the reward of the games. However, NIDs occasionally give 
place to assessment and feedback components in their products. This might have been due to a 
lack of strategies to monitor the players. In a few cases, the designers created questions which 
the players had to answer in order to continue to other parts of the learning environment. 
However, the designers could not provide quality challenges for the players in the 3D setting, 
apart from using some visible/invisible objects and teleports. For example, in the Library project, 
the players were supposed to find three books in the library, but the designers could not find a 
technical solution to control how many books were found by the students. Another problem 
was guiding the players because the AW environment is a huge platform. The novice designers 
provided several guiding objects to make the players follow a correct path to solve problems. 
Since monitoring the users was not possible unless they interacted with the objects, assessing 
their actions and providing feedback was a challenge for the novice designers. Another difficulty 
was that the users could continue their trip in the 3D setting, even if they made mistakes. This 
means that for the novice designers, designing assessment, feedback, punishment, and reward 
systems was not easy in the 3D setting. 

Table 6. Feedback and Rewarding Issues that Arose during the Project 


Challenges to design feedback and rewarding 

Tracing user behavior 
Finishing a scenario 

Thinking of all the possible actions of users to guide them 

Even if users make mistakes they can continue to play without any warning or 

punishment 

Integration of feedback and rewarding 

Using stable objects to provide feedback 
2D feedback 

Using a Website to control the users' behaviors 


In the projects, the most popular feedback method was to provide small text objects to guide 
the users. When users entered a place, they could see a text object that told them what they 
can do or what they should do. The majority of the feedback was provided via the Web 
component. The novice designers also used the visibility and invisibility properties of the objects 
in 3D setting. Thus, they could provide feedback in several formats, by showing colorful objects, 
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requesting that the users undertake another mission, or displaying the progress of the users. To 
halt the progress of users, two groups wanted to provide some key words in the immersive 
world, by which the users could access the Web component. This, some user decisions could be 
facilitated with the help of those key words. However, the novice designers could not ensure 
that the users could see all of the key words at any one time. In this respect, the designers' 
automated assessment systems did not work well. 


Discussion and Conclusion 

Currently, there are many platforms like Active Worlds (AW) that permit 3D virtual world 
development. It can be argued that the novices' expectations were highly influenced by design 
issues relating to platforms with which they were familiar. Therefore, this study's results indicate 
that it is important for novices to experience several different kinds of platforms during their 
training so that they may more readily comprehend the goals and opportunities provided in 
those platforms. 

Hirumi et al. (2010a; 2010b) suggested that in the education of novice instructional designers, 
experiencing the design issues of game-based learning should be a part of their training. Most 
of the novices in this study were only familiar with 3D video games; this might have caused 
expectations from virtual worlds that this technology could not provide. Today, children like 
active and competitive environments for fun play. According to the novice designers, productive 
learning interaction should involve more than only mouse-clicking and reading content, or 
discovering something just by walking around (as an avatar) in the platform. Unless more 
interactivity is provided, one of the major motivational issues faced by the novice designers in 
their projects could not be solved. Novice instructional designers spend less time analyzing the 
problems that they encounter and elaborate different design solutions (Perez & Emery, 1995). 
In this study also the novice designers' lack of applied analysis regarding the 3D part may be 
another reason that they did not use it very efficiently. The novice designers in this study clearly 
wished that real interaction features be provided in the 3D platform, just as they are in video 
games, which requires more complex programming. As Dickey (2006) states, the platform 
should not compel the instructional designers to possess talented artistic skills. 

Monitoring the users was one of the biggest challenges of novice instructional designers. In 
particular, they found it difficult to create a chain of learning conditions, since they could not 
easily monitor the students in the platform. The novice designers did not consider utilizing 
different communication tools to obtain information from the users, but instead tried to use 
automated monitoring systems, which are not usual for this kind of platform. To overcome this 
difficulty, the novice designers would have needed to find different strategies to monitor the 
students, and to increase the communication between the teachers and the students. However, 
the novices' prior experiences did not help them to suggest new strategies for this unfamiliar 
platform. Thus again, instructional design novices might benefit from exposure to a variety of 
different design contexts to improve their contextual design skills. In this study, it was apparent 
that the novices could not use the three main components of the development platform 
effectively. Their designs for the learning environment were based upon their previous 
experiences with 2D development platforms and video games. 

Although virtual worlds do not offer a real game environment, this idea might be taken into 
consideration by platform developers, so that they can design more attractive environments. 
Kirkley and Kirkley (2004) noted that fun can be intertwined with challenges; and virtual worlds 
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are gaining popularity because they are said to be useful for constructivist inquiry and as 
discovery environments (Dickey, 2003; 2005a; 2005b). Therefore, challenges for users might be 
provided in the learning tasks outside the 3D setting of the platform. To provide cognitive 
challenges, immersive virtual environments should be made more flexible in terms of the design 
programming. The novice instructional designers in this study were aware of what might be 
motivating for the users, since their concerns cover both core learning variables and a broad 
range of pedagogical issues which they found to be challenging. Platform developers should 
note these issues as problems that they can remedy in the future. 

One of the responsibilities of instructional designers is to provide a smooth interactivity between 
communication tools, Web-based tools, and the immersive platform. In this study, the novice 
designers tried to do this in simple ways. Since they were designing very short-term learning 
environments, they ignored the student-student and student-teacher communication aspects 
of learning. They also tried to put start and end points in the platform, with a set of guiding 
features for the users. In the end, there were no student-centered environments in any of the 
projects because of this linear structure of the projects. That result revealed two issues relating 
to the use of this kind of platform in design education. The first is that these platforms are more 
suitable for long-term engagement in learning (such as for problem-solving). The second issue is 
that creating a learning environment, which has a linear structure, is very difficult in this kind of 
platforms, since the platform does not limit user actions between a start and end point. As a 
long-term and student-centered learning environment, virtual worlds offer many design 
education opportunities. In this study, the novice designers dealt with many instructional, 
motivational, and technical problems. However, limitations of virtual environments should not 
disrupt confidence and competence of designers in designing these environments (Bower, Cram, 
& Groom, 2010). They changed their stories and scenarios many times to achieve higher user 
motivation. They also assessed the various advantages offered by different 3D settings. Yet, they 
did not enjoy designing virtual worlds as much expected. 

The limitations and opportunities provided by a development platform might influence the 
design process (Schaefer & Warren 2004). In this study, some technical issues were caused by 
the limited number of designer participants, and some of the challenges faced by the novice 
designers might have been caused by the limitations of their background knowledge or the 
quality of their project group work. First of all, most of the participants came to the course 
thinking that they would learn how to develop attractive games. As they encountered the 
provided virtual world environment, their motivation was reduced, because they could not 
accomplish most of their pre-formed ideas. They were not able to use all of the objects that they 
intended to use, and they could not provide the challenges that they wanted to pose for the 
users. They also had difficulty finding objects, since there was no clear organization of the 
objects. 

The premise behind this article might be criticized because instructional designers are not 
supposed to develop the environment technically. However, without technical experience, 
designers cannot imagine how their design will fit into the development platform. Also, in the 
general definitions of competencies in The International Board of Standards for Training, 
Performance and Instruction (IBSTPI) of the essential competencies of instructional designers is 
the ability to develop instructional materials. Moreover, in Turkey, the Instructional Design 
program covers programming and technical skills. Another criticism might be related the reason 
why some novices' had unfeasible designs although there are facilitators guiding them. It should 
also be noted that the facilitators of the project groups in this study reminded the groups to 
investigate the design environment very carefully, and they warned the project groups about 
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the feasibility of their scenarios. However, the participants did not analyze the platform until the 
development stage. This issue is also important to see that novices ignore the important points 
during the analysis process. 

In this study mainly technical part of instructional design was evaluated. To build upon the 
results presented here, further research should be conducted with other phases and of 
instructional design to understand the design challenges and decisions of novices. Since the 
study was conducted in a course context, the participants also had limited time to solve the 
problems faced in their study group projects. Different platforms might also be examined to 
determine whether common problems occur. In conclusion, immersive games like learning 
environments are important platforms which should be investigated by teachers, learners, and 
instructional designers. They provide many decision points which might be facilitated by 
university training programs for future designers. 
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